Final Project: Time Series Forecasting with LSTMs, Neural Networks Eng. Class

prof. Victor H. Alves Ribeiro, PPGEPS/PUCPR

---

---

Note

Chamada do exercicio em ingles vai aqui

Abstract

This project aims to predict the returns of a selected portfolio of agricultural commodities using Long Short-Term Memory (LSTM) neural networks. The study includes ablation experiments to understand the impact of different architectural choices on model performance. The results demonstrate the importance of hyperparameter tuning and regularization in time series forecasting tasks.

Introduction

The use of Artificial Neural Networks (ANNs) for time series forecasting has gained significant traction in recent years. This project focuses on utilizing LSTMs, a type of recurrent neural network capable of handling sequential data efficiently, to predict financial returns of agricultural commodities. A detailed ablation study is conducted to explore various architectural configurations and hyperparameters.

Literature Review

Time series forecasting is critical in finance and economics. Traditional models, such as ARIMA and exponential smoothing, have limitations when dealing with non-linear and complex data. Recent studies emphasize the robustness of LSTM models in capturing temporal dependencies. This project builds on existing research by applying LSTM networks to a unique dataset of agricultural commodity returns.

…see the full paper for this complete section …

Methods

• Collecting commodity price data.
• Calculating logarithmic returns.
• Normalizing the data.
• Training LSTM models with different configurations.
• Performing grid search to optimize hyperparameters.
• Conducting residual analysis to identify uncaptured patterns and issues like autocorrelation or heteroscedasticity.

Results discussion

Data Collection and Preprocessing

Python libs

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Importing necessary libraries
import pandas as pd
import numpy as np
import yfinance as yf
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras.optimizers import Adam
from sklearn.model_selection import ParameterGrid
import psutil
import time
import json
import os
import warnings
warnings.filterwarnings('ignore')

# Additional libraries for residual analysis
from statsmodels.stats.diagnostic import acorr_ljungbox, het_breuschpagan
import statsmodels.api as sm

First of all, we need to check the hardware availability:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Collecting hardware information
def get_system_info():
    system_info = {
        'CPU_cores': psutil.cpu_count(logical=True),
        'CPU_freq_MHz': psutil.cpu_freq().current,
        'Total_RAM_GB': round(psutil.virtual_memory().total / (1024 ** 3), 2),
        'Available_RAM_GB': round(psutil.virtual_memory().available / (1024 ** 3), 2),
        'GPU_info': 'Not available'  # Placeholder, can be expanded with libraries like GPUtil
    }
    return system_info

system_info = get_system_info()
print("System Information:", system_info)

<<<<<<< HEAD

System Information: {'CPU_cores': 12, 'CPU_freq_MHz': 1800.0, 'Total_RAM_GB': 31.69, 'Available_RAM_GB': 14.89, 'GPU_info': 'Not available'}

Loading data:

=======

System Information: {'CPU_cores': 12, 'CPU_freq_MHz': 1800.0, 'Total_RAM_GB': 31.69, 'Available_RAM_GB': 17.49, 'GPU_info': 'Not available'}

Loading data:

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Defining the tickers
tickers = [
    "ZC=F",  # Corn Futures
    "ZW=F",  # Wheat Futures
    "KE=F",  # KC HRW Wheat Futures
    "ZR=F",  # Rough Rice Futures
    "GF=F",  # Feeder Cattle Futures
    "ZM=F",  # Soybean Meal Futures
    "ZL=F",  # Soybean Oil Futures
    "ZS=F"   # Soybean Futures
]

# Downloading price data
print("\nDownloading price data...")
data = yf.download(tickers, start="2015-01-01")['Close']

# Handling missing data
print("\nHandling missing data...")
data.fillna(method='ffill', inplace=True)  # Forward fill
data.dropna(axis=1, how='all', inplace=True)  # Drop columns with all NaNs
data.dropna(axis=0, how='any', inplace=True)  # Drop rows with any NaNs

# Verify data
print("\nData columns and their non-null counts:")
print(data.count())

if data.empty:
    print("Data is empty after cleaning. Exiting.")
    exit()

# Calculating logarithmic returns
returns = np.log(data / data.shift(1)).dropna()

# Verify returns
print("\nReturns DataFrame info:")
print(returns.info())
print(returns.head())

if returns.empty:
    print("Returns DataFrame is empty. Exiting.")
    exit()
    

returns.head() # Showing time series used (without features)

Downloading price data...

[                       0%                       ][************          25%                       ]  2 of 8 completed[******************    38%                       ]  3 of 8 completed[**********************50%                       ]  4 of 8 completed[**********************62%*****                  ]  5 of 8 completed[**********************75%***********            ]  6 of 8 completed[**********************88%*****************      ]  7 of 8 completed[*********************100%***********************]  8 of 8 completed

Handling missing data...

Data columns and their non-null counts:
Ticker
<<<<<<< HEAD
GF=F    2538
KE=F    2538
ZC=F    2538
ZL=F    2538
ZM=F    2538
ZR=F    2538
ZS=F    2538
ZW=F    2538
=======
GF=F    2537
KE=F    2537
ZC=F    2537
ZL=F    2537
ZM=F    2537
ZR=F    2537
ZS=F    2537
ZW=F    2537
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
dtype: int64

Returns DataFrame info:
<class 'pandas.core.frame.DataFrame'>
<<<<<<< HEAD
DatetimeIndex: 2537 entries, 2015-01-05 00:00:00+00:00 to 2025-02-05 00:00:00+00:00
Data columns (total 8 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   GF=F    2537 non-null   float64
 1   KE=F    2537 non-null   float64
 2   ZC=F    2537 non-null   float64
 3   ZL=F    2537 non-null   float64
 4   ZM=F    2537 non-null   float64
 5   ZR=F    2537 non-null   float64
 6   ZS=F    2537 non-null   float64
 7   ZW=F    2537 non-null   float64
dtypes: float64(8)
memory usage: 178.4 KB
=======
DatetimeIndex: 2536 entries, 2015-01-05 00:00:00+00:00 to 2025-02-04 00:00:00+00:00
Data columns (total 8 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   GF=F    2536 non-null   float64
 1   KE=F    2536 non-null   float64
 2   ZC=F    2536 non-null   float64
 3   ZL=F    2536 non-null   float64
 4   ZM=F    2536 non-null   float64
 5   ZR=F    2536 non-null   float64
 6   ZS=F    2536 non-null   float64
 7   ZW=F    2536 non-null   float64
dtypes: float64(8)
memory usage: 178.3 KB
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
None
Ticker                         GF=F      KE=F      ZC=F      ZL=F      ZM=F  \
Date                                                                          
2015-01-05 00:00:00+00:00  0.007673  0.012483  0.025570  0.023203  0.034462   
2015-01-06 00:00:00+00:00 -0.004330  0.010350 -0.002466 -0.000306  0.004866   
2015-01-07 00:00:00+00:00  0.004219 -0.017983 -0.021842  0.008832 -0.006222   
2015-01-08 00:00:00+00:00 -0.000111 -0.019956 -0.005060  0.018029 -0.019732   
2015-01-09 00:00:00+00:00 -0.014284 -0.012001  0.015104 -0.001192  0.006896   

Ticker                         ZR=F      ZS=F      ZW=F  
Date                                                     
2015-01-05 00:00:00+00:00  0.003537  0.036483  0.013245  
2015-01-06 00:00:00+00:00  0.002644  0.010762  0.004658  
2015-01-07 00:00:00+00:00  0.004392  0.001664 -0.020919  
2015-01-08 00:00:00+00:00 -0.010130 -0.007389 -0.021806  
2015-01-09 00:00:00+00:00  0.002653  0.006201 -0.005748  

Ticker	GF=F	KE=F	ZC=F	ZL=F	ZM=F	ZR=F	ZS=F	ZW=F
Date
2015-01-05 00:00:00+00:00	0.007673	0.012483	0.025570	0.023203	0.034462	0.003537	0.036483	0.013245
2015-01-06 00:00:00+00:00	-0.004330	0.010350	-0.002466	-0.000306	0.004866	0.002644	0.010762	0.004658
2015-01-07 00:00:00+00:00	0.004219	-0.017983	-0.021842	0.008832	-0.006222	0.004392	0.001664	-0.020919
2015-01-08 00:00:00+00:00	-0.000111	-0.019956	-0.005060	0.018029	-0.019732	-0.010130	-0.007389	-0.021806
2015-01-09 00:00:00+00:00	-0.014284	-0.012001	0.015104	-0.001192	0.006896	0.002653	0.006201	-0.005748

Plotting the time series of prices and returns side by side (2 per row)

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Create a directory for plots if it doesn't exist
plots_dir = 'plots'
if not os.path.exists(plots_dir):
    os.makedirs(plots_dir)

# Plot prices
print("\nPlotting time series of prices...")
num_cols = 2  # Number of plots per row
num_plots = len(data.columns)
num_rows = (num_plots + num_cols - 1) // num_cols  # Ensure enough rows

fig, axs = plt.subplots(num_rows, num_cols, figsize=(15, 5 * num_rows))
axs = axs.flatten()

for i, col in enumerate(data.columns):
    axs[i].plot(data.index, data[col])
    axs[i].set_title(f'Price Series - {col}')
    axs[i].set_xlabel('Date')
    axs[i].set_ylabel('Price')

# Hide unused subplots
for j in range(i + 1, len(axs)):
    fig.delaxes(axs[j])

plt.tight_layout()
plt.savefig(os.path.join(plots_dir, 'price_series.png'))
plt.show()
plt.close()

# Plot returns
print("Plotting time series of returns...")
num_plots_ret = len(returns.columns)
num_rows_ret = (num_plots_ret + num_cols - 1) // num_cols

fig, axs = plt.subplots(num_rows_ret, num_cols, figsize=(15, 5 * num_rows_ret))
axs = axs.flatten()

for i, col in enumerate(returns.columns):
    axs[i].plot(returns.index, returns[col])
    axs[i].set_title(f'Return Series - {col}')
    axs[i].set_xlabel('Date')
    axs[i].set_ylabel('Log Return')

# Hide unused subplots
for j in range(i + 1, len(axs)):
    fig.delaxes(axs[j])

plt.tight_layout()
plt.savefig(os.path.join(plots_dir, 'return_series.png'))
plt.show()
plt.close()

Plotting time series of prices...

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

Plotting time series of returns...

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

Preprocessing data for LSTM time series modelling:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Function to prepare data for LSTM
def prepare_data(series, time_steps):
    X, y = [], []
    for i in range(len(series) - time_steps):
        X.append(series[i:(i + time_steps)])
        y.append(series[i + time_steps])
    return np.array(X), np.array(y)

Setting the parameters:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Defining parameters
time_steps = 5  # Number of time steps
epochs = 10  # Reduced epochs for faster execution during testing

# Dictionaries to store results
models = {}
histories = {}
mse_results = {}
scalers = {}
predictions = {}
best_params_dict = {}
residuals_analysis = {}

# Directory to save reports and graphs
report_dir = 'report'
if not os.path.exists(report_dir):
    os.makedirs(report_dir)

LSTM time series model fitting

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Loop through each time series
for col in returns.columns:
    print(f"\nProcessing column: {col}")
    series = returns[col].values.reshape(-1, 1)
    
    # Check if series is empty
    if len(series) == 0:
        print(f"Series {col} is empty after preprocessing. Skipping.")
        continue
    
    print(f"Series {col} has {len(series)} data points.")
    
    # Normalizing data
    scaler = MinMaxScaler(feature_range=(0, 1))
    series_scaled = scaler.fit_transform(series)
    scalers[col] = scaler  # Storing the scaler for later inversion
    
    # Preparing data
    X, y = prepare_data(series_scaled, time_steps)
    
    # Check if X and y are non-empty
    if X.shape[0] == 0:
        print(f"Not enough data points in {col} after preparation. Skipping.")
        continue
    
    # Splitting into training and test sets
    split_index = int(0.8 * len(X))
    X_train_full, X_test = X[:split_index], X[split_index:]
    y_train_full, y_test = y[:split_index], y[split_index:]
    X_train_full = X_train_full.reshape((X_train_full.shape[0], X_train_full.shape[1], 1))
    X_test = X_test.reshape((X_test.shape[0], X_test.shape[1], 1))
    
    # Hyperparameter grid for Grid Search
    param_grid = {
        'neurons': [30, 50],
        'learning_rate': [0.001, 0.01],
        'activation': ['tanh', 'relu'],
        'batch_size': [32, 64]
    }
    grid = ParameterGrid(param_grid)
    
    # Initializing variables to store best results
    best_mse = float('inf')
    best_params = None
    best_model = None
    
    # Performing Grid Search
    print(f"Performing Grid Search for {col}...")
    for params in grid:
        model = Sequential()
        model.add(LSTM(params['neurons'], activation=params['activation'], input_shape=(time_steps, 1)))
        model.add(Dense(1))
        optimizer = Adam(learning_rate=params['learning_rate'])
        model.compile(optimizer=optimizer, loss='mean_squared_error')
        
        history = model.fit(
            X_train_full, y_train_full,
            validation_data=(X_test, y_test),
            epochs=epochs,
            batch_size=params['batch_size'],
            verbose=0
        )
        
        y_pred = model.predict(X_test)
        y_pred_inv = scaler.inverse_transform(y_pred)
        y_test_inv = scaler.inverse_transform(y_test.reshape(-1, 1))
        mse = mean_squared_error(y_test_inv, y_pred_inv)
        
        if mse < best_mse:
            best_mse = mse
            best_params = params
            best_model = model
            best_y_pred = y_pred
    
    best_params_dict[col] = best_params
    print(f"Best parameters for {col}: {best_params} with MSE: {best_mse}")
    
    models[col] = best_model
    predictions[col] = {'Best Model': best_y_pred}
    
    # Inverting the normalization
    y_test_inv = scaler.inverse_transform(y_test.reshape(-1, 1))
    y_pred_inv = scaler.inverse_transform(best_y_pred)
    
    # Calculating MSE
    mse_results[col] = {'Best Model': best_mse}
    
    # Visualization of results
    plt.figure(figsize=(10, 4))
    plt.plot(y_test_inv, label='Actual Value')
    plt.plot(y_pred_inv, label='Prediction')
    plt.title(f'Prediction vs Actual - {col} - Best Model')
    plt.legend()
    plt.savefig(os.path.join(report_dir, f'pred_vs_actual_{col}_Best_Model.png'))
    plt.close()
    
    # Residual Analysis
    residuals = y_test_inv - y_pred_inv
    
    # Plotting residuals
    plt.figure(figsize=(10, 4))
    plt.plot(residuals, label='Residuals')
    plt.title(f'Residuals - {col} - Best Model')
    plt.legend()
    plt.savefig(os.path.join(report_dir, f'residuals_{col}_Best_Model.png'))
    plt.close()
    
    # Ljung-Box test for autocorrelation in residuals
    lb_test = acorr_ljungbox(residuals, lags=[10], return_df=True)
    lb_pvalue = lb_test['lb_pvalue'].values[0]
    
    # Plotting residuals ACF
    fig, ax = plt.subplots(figsize=(10, 4))
    sm.graphics.tsa.plot_acf(residuals.squeeze(), lags=40, ax=ax)
    plt.title(f'Residuals Autocorrelation Function - {col}')
    plt.savefig(os.path.join(report_dir, f'acf_residuals_{col}_Best_Model.png'))
    plt.close()
    
    # Heteroscedasticity test (Breusch-Pagan Test)
    exog = sm.add_constant(best_model.predict(X_test))
    test_bp = het_breuschpagan(residuals, exog)
    bp_pvalue = test_bp[3]
    
    # Convert p-values to Python float
    lb_pvalue = float(lb_pvalue)
    bp_pvalue = float(bp_pvalue)
    
    # Saving statistical test results
    residuals_analysis[col] = {
        'residuals': residuals.flatten().tolist(),
        'ljung_box_pvalue': lb_pvalue,
        'breusch_pagan_pvalue': bp_pvalue
    }
    
    print(f"Residual Analysis for {col}:")
    print(f"Ljung-Box Test p-value: {lb_pvalue}")
    print(f"Breusch-Pagan Test p-value: {bp_pvalue}")

# Displaying final results in a table
print("\nFinal Results:")
results_table = pd.DataFrame(mse_results)
print(results_table)

Processing column: GF=F
<<<<<<< HEAD
Series GF=F has 2537 data points.
Performing Grid Search for GF=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 81ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 83ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 94ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 81ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 74ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 107ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 89ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 86ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
Best parameters for GF=F: {'activation': 'relu', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00010924864900025345
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 374us/step
Residual Analysis for GF=F:
Ljung-Box Test p-value: 0.7999419798125964
Breusch-Pagan Test p-value: 0.5169359494108828

Processing column: KE=F
Series KE=F has 2537 data points.
Performing Grid Search for KE=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 90ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 89ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 74ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 89ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 93ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 66ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 78ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 76ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 82ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 77ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
Best parameters for KE=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00038164028806659286
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 0s/step  
Residual Analysis for KE=F:
Ljung-Box Test p-value: 0.020418486696801786
Breusch-Pagan Test p-value: 0.9515329525463874

Processing column: ZC=F
Series ZC=F has 2537 data points.
Performing Grid Search for ZC=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 75ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 81ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 75ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 64ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 78ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 113ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
Best parameters for ZC=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.00030749551565998105
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 547us/step
Residual Analysis for ZC=F:
Ljung-Box Test p-value: 6.013720898116622e-09
Breusch-Pagan Test p-value: 0.6868146276637007

Processing column: ZL=F
Series ZL=F has 2537 data points.
Performing Grid Search for ZL=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 96ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 96ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 121ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 96ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 85ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 100ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 98ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 96ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 74ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 100ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 81ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
Best parameters for ZL=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.0003702602208585623
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 0s/step  
Residual Analysis for ZL=F:
Ljung-Box Test p-value: 0.1298529565603135
Breusch-Pagan Test p-value: 0.004690939099374363

Processing column: ZM=F
Series ZM=F has 2537 data points.
Performing Grid Search for ZM=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 91ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 99ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 83ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 89ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 77ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 78ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 89ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
Best parameters for ZM=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00033244725577178525
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 468us/step
Residual Analysis for ZM=F:
Ljung-Box Test p-value: 0.05553709303469302
Breusch-Pagan Test p-value: 0.17414706671883953

Processing column: ZR=F
Series ZR=F has 2537 data points.
Performing Grid Search for ZR=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 101ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 93ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 86ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 83ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 90ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 67ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 96ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 83ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 90ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 86ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 84ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
Best parameters for ZR=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.001, 'neurons': 50} with MSE: 0.09115055411173459
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 620us/step
Residual Analysis for ZR=F:
Ljung-Box Test p-value: 1.3654299407243602e-19
Breusch-Pagan Test p-value: 1.686896553071609e-08

Processing column: ZS=F
Series ZS=F has 2537 data points.
Performing Grid Search for ZS=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 81ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 77ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 85ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 74ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 86ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 82ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 85ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 88ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
Best parameters for ZS=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00015064847517274975
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step 
Residual Analysis for ZS=F:
Ljung-Box Test p-value: 0.10351190804700801
Breusch-Pagan Test p-value: 0.20285013242058

Processing column: ZW=F
Series ZW=F has 2537 data points.
Performing Grid Search for ZW=F...
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 100ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step  16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 86ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 98ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 16s 1s/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 83ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 78ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 98ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 80ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 72ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step
 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 79ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step
Best parameters for ZW=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.00040139610777335967
 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 456us/step
Residual Analysis for ZW=F:
Ljung-Box Test p-value: 0.03656219242619252
Breusch-Pagan Test p-value: 0.9649894198655294

Final Results:
                GF=F      KE=F      ZC=F     ZL=F      ZM=F      ZR=F  \
Best Model  0.000109  0.000382  0.000307  0.00037  0.000332  0.091151   

                ZS=F      ZW=F  
Best Model  0.000151  0.000401  

Saving the results in a table:

======= Series GF=F has 2536 data points. Performing Grid Search for GF=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 166ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 110ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 151ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 132ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 133ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 134ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 134ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 109ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 111ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 144ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 133ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 142ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 128ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 183ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 98ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step Best parameters for GF=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00010857176433668775 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 17ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step Residual Analysis for GF=F: Ljung-Box Test p-value: 0.9072686287654939 Breusch-Pagan Test p-value: 0.5152988935632297 Processing column: KE=F Series KE=F has 2536 data points. Performing Grid Search for KE=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 100ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 127ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 123ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 132ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 140ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 126ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 112ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 117ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 134ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 101ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 143ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 169ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 117ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 168ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step Best parameters for KE=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.00038118692504776 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step Residual Analysis for KE=F: Ljung-Box Test p-value: 0.023564794649141935 Breusch-Pagan Test p-value: 0.7227360511261485 Processing column: ZC=F Series ZC=F has 2536 data points. Performing Grid Search for ZC=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 129ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 122ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 3s 222ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 172ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 149ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 191ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 152ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 156ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 167ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 146ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 112ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 140ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 95ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 122ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 159ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 135ms/step13/16 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step Best parameters for ZC=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.0003130356815043125 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 22ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step Residual Analysis for ZC=F: Ljung-Box Test p-value: 1.09007955457476e-09 Breusch-Pagan Test p-value: 0.4467170767690787 Processing column: ZL=F Series ZL=F has 2536 data points. Performing Grid Search for ZL=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 150ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 156ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 113ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 114ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 141ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 120ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 131ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 111ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 150ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 116ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 108ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 154ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 123ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 111ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 101ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 116ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step Best parameters for ZL=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.00037079724055945963 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step Residual Analysis for ZL=F: Ljung-Box Test p-value: 0.16637822021822346 Breusch-Pagan Test p-value: 0.0054782637959072235 Processing column: ZM=F Series ZM=F has 2536 data points. Performing Grid Search for ZM=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 136ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 115ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 149ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 162ms/step10/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 124ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 107ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 131ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 139ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 106ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 130ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 130ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 58ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 1s 58ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 132ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 127ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 168ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step Best parameters for ZM=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.00032991523349328066 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 25ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step Residual Analysis for ZM=F: Ljung-Box Test p-value: 0.11496994851434907 Breusch-Pagan Test p-value: 0.47858399886650005 Processing column: ZR=F Series ZR=F has 2536 data points. Performing Grid Search for ZR=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 177ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 139ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 131ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 139ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 153ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 119ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 116ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 114ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 153ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 121ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 114ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 123ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 99ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 126ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 115ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 109ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step Best parameters for ZR=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.001, 'neurons': 50} with MSE: 0.09048076499091248 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step Residual Analysis for ZR=F: Ljung-Box Test p-value: 2.409858724893761e-19 Breusch-Pagan Test p-value: 7.898502568638808e-06 Processing column: ZS=F Series ZS=F has 2536 data points. Performing Grid Search for ZS=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 134ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 193ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 164ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 120ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 106ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 121ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 115ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 118ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 103ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 109ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 113ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 124ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 121ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 7ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 104ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 125ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step Best parameters for ZS=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50} with MSE: 0.00015281614180061257 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 26ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step Residual Analysis for ZS=F: Ljung-Box Test p-value: 0.022861004524120033 Breusch-Pagan Test p-value: 0.11417814352478779 Processing column: ZW=F Series ZW=F has 2536 data points. Performing Grid Search for ZW=F... 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 168ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 132ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 161ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 164ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 135ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 123ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 124ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 131ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 105ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 110ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 122ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 130ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 161ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 1s 116ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 140ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/16 ━━━━━━━━━━━━━━━━━━━━ 2s 144ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 10ms/step Best parameters for ZW=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30} with MSE: 0.0003999852739209394 1/16 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step Residual Analysis for ZW=F: Ljung-Box Test p-value: 0.02007895358014522 Breusch-Pagan Test p-value: 0.9093949646955048 Final Results: GF=F KE=F ZC=F ZL=F ZM=F ZR=F \ Best Model 0.000109 0.000381 0.000313 0.000371 0.00033 0.090481 ZS=F ZW=F Best Model 0.000153 0.0004

Saving the results in a table:

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Saving results to a CSV file
results_table.to_csv(os.path.join(report_dir, 'mse_results_updated.csv'), index=True)

# Saving the best parameters found
with open(os.path.join(report_dir, 'best_params.json'), 'w') as f:
    json.dump(best_params_dict, f, indent=4)

# Saving the residual analysis
with open(os.path.join(report_dir, 'residuals_analysis.json'), 'w') as f:
    json.dump(residuals_analysis, f, indent=4)

Ploting the MSEs for each time series:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Report: Documenting the results
# Plotting the MSEs for each time series
for col in mse_results.keys():
    mse_series = mse_results[col]
    plt.figure(figsize=(10, 5))
    plt.bar(mse_series.keys(), mse_series.values(), color='blue')
    plt.title(f'MSE Comparison - {col}')
    plt.ylabel('MSE')
    plt.xticks(rotation=45)
    plt.tight_layout()
    plt.savefig(os.path.join(report_dir, f'mse_comparison_{col}.png'))
    plt.close()

Saving system info:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# End timer
end_time = datetime.now()
elapsed_time = end_time - start_time  # This is a timedelta object
print(f"Total execution time: {elapsed_time}")

# Save execution time to the report
system_info['Execution_Time_seconds'] = elapsed_time.total_seconds()  # Convert to float for JSON
with open(os.path.join(report_dir, 'system_info.json'), 'w') as f:
    json.dump(system_info, f, indent=4)

<<<<<<< HEAD

Total execution time: 0:04:47.669284

Generating an automatic final report:

=======

Total execution time: 0:07:26.699982

Generating an automatic final report:

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Final Report: Generating a text document with the results
report_path = os.path.join(report_dir, 'final_report.txt')
with open(report_path, 'w') as report_file:
    report_file.write("Final Project Report - Forecasting Commodity Returns with LSTM\n")
    report_file.write("="*80 + "\n\n")
    
    report_file.write("1. Project Objectives:\n")
    report_file.write("Forecast future returns of a commodity portfolio using LSTM Neural Networks.\n\n")
    
    report_file.write("2. Methodology:\n")
    report_file.write("- Collecting commodity price data.\n")
    report_file.write("- Calculating logarithmic returns.\n")
    report_file.write("- Normalizing the data.\n")
    report_file.write("- Training LSTM models with different configurations.\n")
    report_file.write("- Performing grid search to optimize hyperparameters.\n")
    report_file.write("- Conducting residual analysis to identify uncaptured patterns and issues like autocorrelation or heteroscedasticity.\n\n")
    
    report_file.write("3. Results:\n")
    report_file.write(results_table.to_string())
    report_file.write("\n\n")
    
    report_file.write("4. Best Parameters Found (Grid Search):\n")
    report_file.write(json.dumps(best_params_dict, indent=4))
    report_file.write("\n\n")
    
    report_file.write("5. Residual Analysis:\n")
    for col, res in residuals_analysis.items():
        report_file.write(f"Residual Analysis for {col}:\n")
        report_file.write(f"Ljung-Box Test p-value: {res['ljung_box_pvalue']}\n")
        report_file.write(f"Breusch-Pagan Test p-value: {res['breusch_pagan_pvalue']}\n\n")
    report_file.write("\n")
    
    report_file.write("6. Conclusions:\n")
    report_file.write("The study demonstrated the importance of proper hyperparameter selection and model architecture for forecasting financial returns. Regularization techniques and the choice of activation function significantly influenced model performance. The residual analysis highlighted the need to consider autocorrelation and heteroscedasticity in modeling financial time series.\n\n")
    
    report_file.write("7. Recommendations for Future Work:\n")
    report_file.write("- Implement additional regularization techniques, such as DropConnect or Batch Normalization.\n")
    report_file.write("- Explore more advanced architectures, like GRU or bidirectional models.\n")
    report_file.write("- Increase the dataset to improve the models' generalization capacity.\n")
    report_file.write("- Use more robust cross-validation methods to assess model stability.\n")
    report_file.write("- Integrate other features, such as technical indicators or macroeconomic variables, to enrich model inputs.\n")
    report_file.write("- Consider hybrid models that combine Machine Learning techniques with traditional statistical models.\n")
    
    report_file.write("\nSystem Information and Execution Time:\n")
    report_file.write(json.dumps(system_info, indent=4))
    report_file.write("\n\n")
    
    report_file.write("End of Report.\n")

Results and Discussion

We begin reading the stored results by MSEs:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# Reading the stored results

# 1. Reading the MSE results file
# Define the report directory
report_dir = 'report'

# Path to the MSE results file
mse_results_path = os.path.join(report_dir, 'mse_results_updated.csv')

# Read the CSV file
mse_results = pd.read_csv(mse_results_path, index_col=0)

# Display the DataFrame
print("\nMSE Results of the models:")
print(mse_results)

MSE Results of the models:
<<<<<<< HEAD
                GF=F      KE=F      ZC=F     ZL=F      ZM=F      ZR=F  \
Best Model  0.000109  0.000382  0.000307  0.00037  0.000332  0.091151   

                ZS=F      ZW=F  
Best Model  0.000151  0.000401  

Then we go to the best hyperparametes file for each time series univariate modelling:

======= GF=F KE=F ZC=F ZL=F ZM=F ZR=F \ Best Model 0.000109 0.000381 0.000313 0.000371 0.00033 0.090481 ZS=F ZW=F Best Model 0.000153 0.0004

Then we go to the best hyperparametes file for each time series univariate modelling:

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# 2. Reading the best parameters file
# Path to the best parameters file
best_params_path = os.path.join(report_dir, 'best_params.json')

# Read the JSON file
with open(best_params_path, 'r') as f:
    best_params = json.load(f)

# Display the best parameters
print("\nBest parameters found for each time series:")
for series, params in best_params.items():
    print(f"{series}: {params}")

Best parameters found for each time series:
<<<<<<< HEAD
GF=F: {'activation': 'relu', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50}
KE=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50}
ZC=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30}
ZL=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50}
ZM=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50}
ZR=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.001, 'neurons': 50}
ZS=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50}
=======
GF=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 50}
KE=F: {'activation': 'relu', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30}
ZC=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50}
ZL=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30}
ZM=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30}
ZR=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.001, 'neurons': 50}
ZS=F: {'activation': 'tanh', 'batch_size': 64, 'learning_rate': 0.01, 'neurons': 50}
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
ZW=F: {'activation': 'tanh', 'batch_size': 32, 'learning_rate': 0.01, 'neurons': 30}

And then we read the final report:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# 3. Reading the final report
# Path to the final report
report_path = os.path.join(report_dir, 'final_report.txt')

# Read the report
with open(report_path, 'r') as report_file:
    report_content = report_file.read()

# Display the report
print("\nFinal Report Content:")
print(report_content)

Final Report Content:
Final Project Report - Forecasting Commodity Returns with LSTM
================================================================================

1. Project Objectives:
Forecast future returns of a commodity portfolio using LSTM Neural Networks.

2. Methodology:
- Collecting commodity price data.
- Calculating logarithmic returns.
- Normalizing the data.
- Training LSTM models with different configurations.
- Performing grid search to optimize hyperparameters.
- Conducting residual analysis to identify uncaptured patterns and issues like autocorrelation or heteroscedasticity.

3. Results:
<<<<<<< HEAD
                GF=F      KE=F      ZC=F     ZL=F      ZM=F      ZR=F      ZS=F      ZW=F
Best Model  0.000109  0.000382  0.000307  0.00037  0.000332  0.091151  0.000151  0.000401
=======
                GF=F      KE=F      ZC=F      ZL=F     ZM=F      ZR=F      ZS=F    ZW=F
Best Model  0.000109  0.000381  0.000313  0.000371  0.00033  0.090481  0.000153  0.0004
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

4. Best Parameters Found (Grid Search):
{
    "GF=F": {
        "activation": "relu",
<<<<<<< HEAD
        "batch_size": 64,
=======
        "batch_size": 32,
        "learning_rate": 0.01,
        "neurons": 50
    },
    "KE=F": {
        "activation": "relu",
        "batch_size": 32,
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
        "learning_rate": 0.01,
        "neurons": 50
    },
<<<<<<< HEAD
    "KE=F": {
        "activation": "relu",
        "batch_size": 32,
        "learning_rate": 0.01,
        "neurons": 50
    },
    "ZC=F": {
        "activation": "tanh",
        "batch_size": 32,
=======
    "ZC=F": {
        "activation": "tanh",
        "batch_size": 64,
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
        "learning_rate": 0.01,
        "neurons": 30
    },
    "ZL=F": {
        "activation": "tanh",
        "batch_size": 32,
        "learning_rate": 0.01,
        "neurons": 50
    },
    "ZM=F": {
        "activation": "relu",
        "batch_size": 32,
        "learning_rate": 0.01,
        "neurons": 30
    },
    "ZR=F": {
        "activation": "tanh",
        "batch_size": 64,
        "learning_rate": 0.001,
        "neurons": 50
    },
    "ZS=F": {
        "activation": "tanh",
        "batch_size": 64,
        "learning_rate": 0.01,
        "neurons": 50
    },
    "ZW=F": {
        "activation": "tanh",
        "batch_size": 32,
        "learning_rate": 0.01,
        "neurons": 30
    }
}

5. Residual Analysis:
Residual Analysis for GF=F:
<<<<<<< HEAD
Ljung-Box Test p-value: 0.7999419798125964
Breusch-Pagan Test p-value: 0.5169359494108828

Residual Analysis for KE=F:
Ljung-Box Test p-value: 0.020418486696801786
Breusch-Pagan Test p-value: 0.9515329525463874

Residual Analysis for ZC=F:
Ljung-Box Test p-value: 6.013720898116622e-09
Breusch-Pagan Test p-value: 0.6868146276637007

Residual Analysis for ZL=F:
Ljung-Box Test p-value: 0.1298529565603135
Breusch-Pagan Test p-value: 0.004690939099374363

Residual Analysis for ZM=F:
Ljung-Box Test p-value: 0.05553709303469302
Breusch-Pagan Test p-value: 0.17414706671883953

Residual Analysis for ZR=F:
Ljung-Box Test p-value: 1.3654299407243602e-19
Breusch-Pagan Test p-value: 1.686896553071609e-08

Residual Analysis for ZS=F:
Ljung-Box Test p-value: 0.10351190804700801
Breusch-Pagan Test p-value: 0.20285013242058

Residual Analysis for ZW=F:
Ljung-Box Test p-value: 0.03656219242619252
Breusch-Pagan Test p-value: 0.9649894198655294
=======
Ljung-Box Test p-value: 0.9072686287654939
Breusch-Pagan Test p-value: 0.5152988935632297

Residual Analysis for KE=F:
Ljung-Box Test p-value: 0.023564794649141935
Breusch-Pagan Test p-value: 0.7227360511261485

Residual Analysis for ZC=F:
Ljung-Box Test p-value: 1.09007955457476e-09
Breusch-Pagan Test p-value: 0.4467170767690787

Residual Analysis for ZL=F:
Ljung-Box Test p-value: 0.16637822021822346
Breusch-Pagan Test p-value: 0.0054782637959072235

Residual Analysis for ZM=F:
Ljung-Box Test p-value: 0.11496994851434907
Breusch-Pagan Test p-value: 0.47858399886650005

Residual Analysis for ZR=F:
Ljung-Box Test p-value: 2.409858724893761e-19
Breusch-Pagan Test p-value: 7.898502568638808e-06

Residual Analysis for ZS=F:
Ljung-Box Test p-value: 0.022861004524120033
Breusch-Pagan Test p-value: 0.11417814352478779

Residual Analysis for ZW=F:
Ljung-Box Test p-value: 0.02007895358014522
Breusch-Pagan Test p-value: 0.9093949646955048
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72


6. Conclusions:
The study demonstrated the importance of proper hyperparameter selection and model architecture for forecasting financial returns. Regularization techniques and the choice of activation function significantly influenced model performance. The residual analysis highlighted the need to consider autocorrelation and heteroscedasticity in modeling financial time series.

7. Recommendations for Future Work:
- Implement additional regularization techniques, such as DropConnect or Batch Normalization.
- Explore more advanced architectures, like GRU or bidirectional models.
- Increase the dataset to improve the models' generalization capacity.
- Use more robust cross-validation methods to assess model stability.
- Integrate other features, such as technical indicators or macroeconomic variables, to enrich model inputs.
- Consider hybrid models that combine Machine Learning techniques with traditional statistical models.

System Information and Execution Time:
{
    "CPU_cores": 12,
    "CPU_freq_MHz": 1800.0,
    "Total_RAM_GB": 31.69,
<<<<<<< HEAD
    "Available_RAM_GB": 14.89,
    "GPU_info": "Not available",
    "Execution_Time_seconds": 287.669284
=======
    "Available_RAM_GB": 17.49,
    "GPU_info": "Not available",
    "Execution_Time_seconds": 446.699982
>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72
}

End of Report.

At the end we read the graphs for MSEs:

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

# 4. Viewing the graphs
from IPython.display import Image, display

# List of time series
series_list = returns.columns

# Display MSE comparison graphs
for col in series_list:
    image_path = os.path.join(report_dir, f'mse_comparison_{col}.png')
    if os.path.exists(image_path):
        display(Image(filename=image_path))
    else:
        print(f"Graph {image_path} not found.")

# Display residuals graphs
for col in series_list:
    residuals_image_path = os.path.join(report_dir, f'residuals_{col}_Best_Model.png')
    acf_image_path = os.path.join(report_dir, f'acf_residuals_{col}_Best_Model.png')
    if os.path.exists(residuals_image_path):
        display(Image(filename=residuals_image_path))
    else:
        print(f"Graph {residuals_image_path} not found.")
    if os.path.exists(acf_image_path):
        display(Image(filename=acf_image_path))
    else:
        print(f"Graph {acf_image_path} not found.")

# End of code

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

Conclusions

 

 

---

References

---

<<<<<<< HEAD

=======

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72

##| eval: false
# Total timing to compile this Quarto document

end_time = datetime.now()
time_diff = end_time - start_time

print(f"Total Quarto document compiling time: {time_diff}")

<<<<<<< HEAD

Total Quarto document compiling time: 0:04:47.981746

=======

Total Quarto document compiling time: 0:07:27.189832

>>>>>>> 5c21489a920a0390dacbd60938e6959145ac8e72